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1.0 Introduction 

The principal objectives of this project are 

1) to develop suitable validation data sets to evaluate the 
effectiveness of the International Satellite Cloud Climatology 
Project (ISCCP) operational algorithm for cloud retrieval in polar 
regions and to validate model simulations of polar cloud cover; 

2) to identify limitations of current procedures for varying 
atmospheric surface conditions , and to explore potential means to 
remedy them using textural classifiers; and 

3) to compare synoptic cloud data from a control run experiment of 
the GISS climate model II with typical observed synoptic cloud 
patterns . 

In the first six months of the first project year a methodology was 
developed for combining AVHRR and SMMR data. These data were calibrated and 
registered to a polar sterographic projection for subsequent digital analysis 
(Maslanik et al., in press). Cloud cover and surface types were manually 
interpreted, viewing angle effects were examined, and the development of a 
catalog of spectral and textural properties of polar clouds and surfaces was 
begun. Additionally, cloud analysis and clustering methods such as Coakley and 
Bretherton's spatial coherence and the fuzzy sets approach were implemented (Key 
et al . , in press) . 

In the second half of the first year, work continued on the catalog of 
spectral and textural features , where new combinations of the calibrated AVHRR 
data (i.e. ratios and differences of AVHRR channels) were included in the 
analysis and data from two seven- day sequences of imagery for two areas of the 
Arctic were studied. The spatial coherence method was extended to include time 
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("temporal coherence") and space and time ("spatiotemporal coherence"). These 
methods were examined to gain further insight into the spatial and temporal 
features of polar clouds and surfaces as they effect the functionality of the 
ISCCP algorithm. Also in the first year, the relationship between theoretical 
and empirical approaches was examined with the GISS general circulation model, 
where the spatial and temporal distributions of monthly average cloud fraction 
were compared to observed cloud amounts. 

This report details the current investigations underway and summarizes the 
progress made to date since the last semi-annual report. The focus has been on 
the implementation and testing of the basic ISCCP cloud detection algorithm for 
use with polar data. 


2.0 ISCCP 

The ISCCP, a project to map clouds with satellite data, began its 
operational activities in July 1983. ISCCP will provide a uniform global 
climatology of the satellite-measured radiances and derive an experimental 
climatology of cloud radiative properties from these radiances. Leading to the 
development of the ISCCP algorithm, Rossow et al . (1985) compared six cloud 
algorithms. The results showed that the performance of all algorithms depends 
on how accurately the clear sky radiances are specified. The ISCCP algorithm 
now being developed is composed of a series of steps, each of which is designed 
to detect some of the clouds present in the scene. The current state of the 
project is such that there is no single version of the algorithm which can be 
applied to all areas of the globe. The algorithm is currently operational for 
low latitude data, but performs rather poorly at high latitudes (Rossow, 1987). 
For this reason, the structure of the ISCCP algorithm is maintained in the cloud 
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analysis procedure presented here, and much of the research described in the 
following chapters is aimed at the modification and calibration of the algorithm 
for use with polar data. At the time of this writing, global results of the 
ISCCP algorithm are available (as the Pilot Climate Data System Cl data set) for 
only July, 1983. 

The cloud detection procedure of the ISCCP cloud algorithm was applied to 
satellite observations of the polar regions by Rossow (1987). It was found that 
the method in general detected too much cloudiness, in part because the algorithm 
does not distinguish between open water/sea ice and snow- covered/snow- free land, 
and because thresholds were not "tuned" for the small temperature differences 
and generally low IR radiances common in the polar regions. No further work on 
the ISCCP algorithm applied to polar data has been reported in the literature. 

2.1 Images Used in Analyses 

In order to adapt the ISCCP algorithm for use with polar data, three areas 
of the Arctic were examined (Figure 1). Study area 1 is centered on the Kara 
and Barents Sea extending north to the pole and south to Norway and the Siberian 
coast. Study area 2 covers most of the Canadian Archipelago and Greenland, and 
extends north to the pole. Study area 3 extends from the coast of Norway to 
Ellesmere Island. Summer and winter images are available for each location. 
A seven-day summer series (July 1-7, 1984) of areas 1 and 2, and a winter series 
(January 6-12, 1984) of area 3 were examined. 

While these areas cover only one-third of the Arctic Basin, they include 
representative samples of all surface types found in the Arctic: snow-covered 
and snow- free land, sea ice of varying concentrations, open water, and permanent 
ice cap. In fact, these areas present particularly difficult conditions for 
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cloud algorithms during the July period; sea ice is moving, snow is melting and 
ponds form, and the extensive coastlines exhibit mixed temperature regimes. 
Reflectances in the study areas were found to vary significantly over the short 
time period and temperatures gradients from North to South were observed. 

These conditions are usual for summer in the Arctic, as are the pressure 
patterns which occurred. Surface pressure maps constructed from Arctic Ocean 
buoy data provide an overall synoptic picture of daily weather. Serreze and Barry 
(1988) found that cyclones are uncommon in the eastern portion of the Canadian 
Archipelago in the summer, but both cyclones and anticyclones are common in and 
around the Barents Sea. Similar conditions can be seen in the data, so that this 
time period may be considered as exhibiting "typical" Arctic summer conditions. 
Although correlations have been determined between synoptic pressure systems, 
cloud amount, and cloud type (Barry et al., 1987), detailed cloud climatologies 
for the Arctic are not available and it is therefore more difficult to make such 
a statement concerning cloud cover. Conditions during the January study period 
are also similar to the mean pattern and are considered to be typical. 

2.2 The ISCCP Algorithm Description 

All cloud algorithms consist of two basic steps: cloud detection and cloud 
analysis. The first step partitions the observed radiances into those 
representing clouds and those that are clear. In the second step, the 
quantitative determination of cloud properties is made from the measured 
radiances. This may involve simply counting cloudy image pixels to obtain a 
single cloud parameter such as fractional cloud cover, or the process may utilize 
radiative transfer models to obtain a parameterized set of cloud properties. 
The ISCCP cloud algorithm has three major components: cloud detection, 
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radiative analysis, and statistical analysis (Rossow et al. , 1985; Rossow, 1987). 
Of primary concern here is the cloud detection step, with limited attention being 
devoted to radiative analysis. The algorithm assumptions are that cloud scene 
radiances are more variable in time and space than clear scene radiances and 
cloudy scenes are associated with larger visible channel and smaller infrared 
radiances than clear scenes. Of particular interest in the cloud detection phase 
is the importance of spatial and temporal variability. The major steps of the 
basic algorithm are summarized in Figure 2 and are: 

Spatial variation . The image is divided into cloudy and clear 
categories based on cold and warm pixels. If a pixel is much colder than the 
warmest pixel in a small region ((100 km) 2 over land and (300 km) 2 over ocean), 
it is label as "cloud 1 * . Otherwise, it is labeled "undecided". High and middle 
level clouds are identified. Only thermal data are used in this step. 

Temporal variation . Pixels are compared to the day before and after 
for changes in temperature. If the middle day is much colder than either the 
day before or the day after, is it is labeled "cloud". If the variation is 
small, it is labeled "clear". Those pixels exhibiting intermediate variability 
are labeled "undecided". Again, only thermal data are used in this step. High 
and middle clouds are most easily recognized. The class of a pixel based on these 
two steps is given in the following truth table: 

| Temporal Variation 
Spatial I 

Variation j Cloud Undecided Clear 


Cloud | Cloud Cloud Mixed 

Undecided | Cloud Undecided Clear 
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Compositing . The mean and extremum radiances for the pixels labeled 
"clear" from the first two steps are calculated over 5 and 30 day periods. 
Statistics are calculated for a 3x3 cell centered on the pixel of interest over 
the time period (i.e., 45 pixels are used for the 5 day period, and 270 for the 
30 day period; this cell hereafter referred to as the "compositing cell"). Note 
that this cell size is acceptable from the point of view of spatial 
autocorrelation ranges, as discussed in section 4.1. The number and mean of 
clear pixels only is recorded as well as the maximum temperature and minimum 
albedo for all pixels (regardless of label) . 

Clear sky radiance map . The clear sky radiances for the pixels which 
are not variable are determined for 5 day periods. Variability here is 
determined by a series of statistical tests, after which the clear sky value may 
be the mean value over the period, the extremum (minimum visible and maximum 
thermal) , or undetermined. 

Bispectral threshold . The data are compared to the clear sky values 
modified by a threshold amount. Those that differ in either the thermal or 
visible channels by more than the threshold amount are labeled as cloud. 

A thoroughly- tested version of the algorithm for low- and mid- latitudes 
has been applied to polar data by Rossow (1987). The algorithm misses 
approximately 20% of the cloud (Rossow, 1987, personal communication). Problems 
cited include: "real" cloud amount is high which forces the algorithm to use more 
extreme values for clear sky radiances; no distinction is made between snow- 
covered and snow- free land, or sea ice and open water in setting thresholds; 
thresholds were not tuned to the low radiances encountered in polar regions; and 
no adjustments were made for solar zenith angle. 
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2.3 Modifications 

The algorithm has been adjusted here in order to deal with these problems. 
The adjustments are: 

1. Data are corrected for solar zenith angle. 

2. Subregion sizes in step 1, classification, are set to 100 km on 
a side for all surface types as suggested by the coherence and 
autocorrelation analyses. 

3. A number of modifications take account of the variety and 
temporal variability of surface types. 

4. 30-day periods are not used in the compositing step. Only five- 
day means and extremum are considered. 

5. Missing values in the clear sky maps are filled based on 
neighboring pixels and the autocorrelation functions. 

6. AVHRR channels 1 and 3 are also employed for the detection of 
low cloud over ice and snow. 

7. If the statistical tests during compositing fail, the clear sky 
value determined for a given location is assigned a value based on 
its spatial neighbors or class characteristic value. 

8. The bispectral threshold step uses channel 3 for ice and snow 
rather than channel 1. 

Some of these modifications require further explanation. 

2.3.1 Spatial and Temporal Variation Tests 

In the spatial variation test of the initial classification, pixels are 
labeled "cloudy" if they are colder than the warmest pixel in the subregion by 
the amounts specified in Table 1. Subregions sizes suggested by Rossow (1987) 
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for this spatial variation test are "about 100 km over land and 300 km over 
ocean". However, five different surface types (snow-covered and snow-free land, 
ocean, sea ice, coast) may occur within subregions of these suggested sizes, 
(300 km) 2 areas of open water are rare, and the autocorrelation analysis 
presented previously would suggest that these sizes are too large. Therefore, 
subregions of size (100 km) 2 (20 x 20 pixel cells) were used regardless of the 
underlying surface type. 


Table 1 

Spatial and temporal thresholds for the three AVHRR 
channels and each surface type . Values for land and 
ocean are from original ISCCP specifications; those 
for other surfaces were derived experimentally. 
Values are in Kelvin. 



LAND 

OCEAN 

ICE 

SNOW 

Spatial 

6.5 

3.5 

3.5 

4.0 

Temporal 

Cloud 

00 

o 

3.5 

5.0 

7.0 

Clear 

Ch. 1 

4.4 

1.4 

8.8 

3.8 

Ch. 3 

5.0 

4.5 

4.5 

4.5 

Ch. 4 

2.5 

1.1 

2.0 

2.0 


In the temporal variation test of the initial classification, where pixel 
temperatures are compared to the day before and after, if a pixel is much colder 
than either day by the amount in Table 1, then that pixel is labeled "cloud". 
If the temperature is the same as either the preceding or following day (by the 
amount also shown in Table 1, 'Clear') then the pixel is labeled "clear". 
Otherwise, it is labeled "undecided". Obvious problems occur when warm, low 
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clouds move into or out of a region where the surface temperature is within the 
"clear" range of the cloud. These cloudy pixels will consequently be labeled 
clear in this step, and were probably labeled "undecided" in the spatial 
variation step because of their relatively high temperature. In this case, they 
will ultimately be labeled clear and used in the compositing step to determine 
clear sky radiances. Since these IR tests fail to label these pixels correctly, 
channel 1 and channel 3 data were also used in the temporal variation test. 

Two other spatial/temporal approaches to the initial classification were 
tested. Rather than comparing each day to the day before and after, the warmest 
pixel in the seven days of data for each location was found. However, the 
warmest pixel in the period was often low cloud, and if the pixel under 
examination was also low cloud, even tests incorporating channels 1 and 3 may 
find little variability and the pixel will be mislabeled as clear. A test of 
each pixel against the warmest in the subregion for any day in the period (and 
of the same surface type) was also performed. Even if a further requirement 
that the warmest also have the lowest albedo is included, periods with 
persistent cloud cover often identify low cloud as the warmest and darkest. 

2.3.2 Compositing 

In the algorithm specifications (Rossow, 1987) the compositing step is to 
use 5 -day means if there are enough clear pixels, or 30-day means or extremum 
otherwise. In the polar regions, surface characteristics may be too variable 
for 30 -day values to be reasonable, particularly in the mid to late summer when 
snow may occur and sea ice moves and changes in concentration. Therefore, it 
is assumed that pixels within a reasonably small spatial area on the same day 
and of the same surface type will be no less similar than the same pixel up to 
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thirty days later. 

If the statistical tests during compositing fail, the clear sky value for 
a given location is assigned a value based on its spatial neighbors or class 
characteristic value. The neighborhood of pixels with the same surface type is 
searched and the first clear value found for a pixel of the same surface type 
is used. The maximum radius is determined by an autocorrelation function (up 
to a radius of 12 pixels). If no value is found within this radius, the clear 
sky value assigned is based on the class characteristic values: the channel 1 
value assigned is the mean for the surface type minus one standard deviation, 
the channel 3 value is the class mean, and the channel 4 value is the mean plus 
one standard deviation. These channel 1 and 4 values correspond to extrema. 
Channel 3 is more difficult as the relationship between brightness temperature 
for cloud and surfaces changes with particle size and physical temperature. 

2.3.3 Surface Types 

Surface types are determined with a land/permanent ice cap mask, SMMR 
data, and SMMR-derived sea ice concentration. The determination of land/not 
land is made with the land mask. If the pixel is land, then a SMMR test is 
applied to determine if the land is snow-free or snow-covered. Snow-covered 
land exhibits a higher brightness temperature at 18 GHz than at 37 GHz (e.g., 
Schweiger, 1987). The vertical polarization is less variable than the 
horizontal for land (unless wet). Hence, if the pixel is land and the 18 GHz 
vertical polarization brightness temperature is higher than in the 37 GHz 
vertical channel, it is labeled snow. This relationship may not hold over an 
ice cap, so a mask for permanent ice cap (e.g., Greenland and Novaya Zemlya) is 
included. Ice cap is labeled snow because of similar albedos. Problems with 
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this method also occur in coastal areas where this relationship may be observed 
even without the presence of snow. Therefore a coastal zone extending 
approximately 20 km from the coastline both inland and seaward is defined. 
Finally, if the pixel is not land and if the sea ice concentration is less than 
15%, the pixel is labeled water, otherwise it is sea ice. 

A problem occurs when the warmest pixel in a subregion is of one surface 
type and the pixel under examination is of another. For example, if the warmest 
pixel is clear sky over land and the pixel under examination is clear sky over 
sea ice, it is likely that the temperature difference will exceed the threshold 
and the sea ice pixel will be labeled cloud. Therefore, a "warmest" pixel for 
each surface type within a subregion is determined. Similarly, only those 
pixels in the compositing cell that have the same surface type as the center 
pixel are counted in the determination of mean and extremum values. 

The basic ISCCP algorithm assumes a constant surface type over the five- 
day period. In most cases, this assumption is valid. However, snow melt in the 
spring and snowfall in late summer are particular problems for this algorithm. 
Perhaps equally likely is ice advection into or out of a region. In both cases, 
emissivities of the surface change, thereby affecting the response in the 
thermal channels. Additionally, albedos change dramatically so that those 
portions of the algorithm which incorporate visible data will be affected the 
most. The problem lies in the determination of the clear sky composite maps for 
the visible and thermal channels. If, for example, sea ice moved into a region 
on the last day of the period, the average albedo making up the clear sky value 
would be significantly affected. This is illustrated in Figure 3, which 
consists of cloud mapped with the AVHRR data and surface types identified using 
SMMR (18 and 37 GHz) and SMMR-derived sea ice concentrations. Cloud cover and 
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surface types are shown for the first day of the seven- day analysis period. 
Surface types are again shown for the sixth day. The change in ice extent to 
the northeast and south of Novaya Zemlya, and the associated change in surface 
albedo and temperature that will in turn affect the spectral responses of 
overlying cloud, illustrate problems that will limit the applicability of the 
ISCCP algorithm in areas with snow and ice cover. If the algorithm detected 
this problem and instead chose an extremum (minimum albedo) for the composite 
value from the first four days, this last day pixel would be labeled "cloud" in 
the final bispectral threshold test. Therefore, pixels in which the surface 
changes during the period are flagged, and do not receive a clear sky value in 
the compositing step. A possible solution is to determine clear sky values for 
each surface type at a location, using data from a 30-day period. 

2.3.4 Statistical Tests 

In order to provide a "population" against which to test compositing cell 
statistics, class characteristic values (i.e., mean and standard deviation) for 
each surface type are computed and updated with each region analyzed. These 
values are initially set to those determined for the previous 5 -day period, or 
from training areas if no previous data are available. Those clear sky composite 
mean values which pass the statistical tests are incorporated into the new class 
characteristic values. 

The statistical tests are designed to determine the likelihood that the 
clear pixels in each compositing cell are in fact all clear. This is done by 
examining the mean, standard deviation, and extremum of the cell. Channel 3 was 
used in addition to channel 4 for snow and ice surfaces. The procedure followed 
here is first to check the number of clear pixels in the compositing cell 
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(maximum 45). The cutoff value for too few pixels is a sample size, n, such 
that we could predict the population mean from the sample mean to within one 
population standard deviation (arbitrary), a, at the 1-a confidence level, i.e., 


E 

where E - a is the maximum error. At a level of significance, a, of 0.01, this 
criterion requires that sample statistics and further tests be based on at least 
six clear pixels. If n is less than six then the maximum thermal and minimum 
visible values are used in the clear sky composite, assuming that the 
probability of them coming from the appropriate populations is greater than the 
specified a-level. 

If, on the other hand, the number of clear pixels is sufficient, the 
probability that the minimum thermal and maximum visible values come from the 
population is tested. If the probability of obtaining either a smaller thermal 
or larger visible value is less than alpha, cloud contamination is assumed and 
the opposite extrema are used as the clear sky composite values. Otherwise, a 
t-test is perfomed on the means of the composite cell and the class 
characteristic values where the null hypothesis is that the means of the 
respective populations are equal (or the difference is 0) . The calculated test 
value is 


t = 


x i 


X 2 


°E 


where a E is the standard error of the estimate: 
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(n 1 -l)s 2 1 + (n 2 -l)s 2 2 1 1 

*.-[ ] 1/2 [--- + --] 1 ' 2 

nj + n 2 - 2 n x n 2 


if the population variances are equal or 


[— + 


| 1/2 


n, n 9 


otherwise, with degrees of freedom of n 1 +n z -2 in both cases. The equality of 
variances is tested with the F statistic, which is the ratio of the two sample 
variances, s 1 i . If the null hypothesis in both tests is not rejected, then the 
mean values are used as the clear sky composite. Otherwise, extrema are used. 


2.3.5 Thresholds and the Distribution of Cloud/Surface Differences 

The final thresholding step utilizes AVHRR channels 1, 3, and 4. Channel 
3 is used only if the surface is sea ice or snow/ice cap, and is intended to 
detect low cloud. Of course, middle and high clouds will be detected over any 
surface with thermal data alone. Thresholds for this step are given in Table 
2. These thresholds are relatively large so that the algorithm yields a rather 
conservative estimate of cloud fraction. 
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Table 2 

Final Thresholds for the three AVHRR channels and each surface type. 


Channel 

LAND 

OCEAN 

ICE 

SNOW 

i 

6.0 

3.5 

6.0 

4.0 

2 

6.0 

6.0 

5.0 

5.0 

3 

8.0 

3.0 

4.0 

4.0 


In this final cloud detection step, a pixel is labeled cloud if it varies 
from the clear sky value by more than the threshold in either channel. The 
importance of this disjunction is illustrated in Figure 4 where the differences 
between cloudy pixel radiances and the radiance of the underlying surface - 
taken to be the clear sky composite value - are plotted for each channel. The 
data are based on samples taken from a number of regions containing a variety 
of surface types and cloud distributions. Zero differences are found along the 
line in each plot. Of particular interest are the points near this line, 
representing optically thin clouds over ice or snow in channel 1, and low 
(possibly inversion) clouds in the channel 4 plot. Those near the zero 
difference line in the channel 3 plot are not clouds with channel 4 temperatures 
similar to the surface, as these tend to exhibit higher brightness temperatures 
than the underlying surface in this channel. A few examples of this situation 
appear as points above the line. These clouds have been found to have 
temperatures and albedos similar to underlying snow and ice surfaces , 
particularly near the North Pole and over Greenland. Of course, a further 
problem is that the difference between channel 3 temperatures for this cloud 
type would be less for locations where the extremum thermal value is used in the 
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composite rather than the mean. 

2.4 Synthetic Data Sets 

In order to test the sensitivity of the components of the ISCCP algorithm, 
a control data set with known characteristics was needed. This section 
describes the development of a synthetic data set which consists of seven days 
of AVHRR data (channels 1, 3, 4), three days of SMMR data (every other day; 18 
and 37 GHz vertical polarization) and sea ice concentration, and a land mask. 
Data sets were created as both a simple geometric representation of clouds and 
surfaces with no internal spectral variation and a free -form model with spectral 
variation controlled by one or more theoretical probability distributions. 

The procedure followed was to first generate the surface and cloud type 
maps for each day of the seven day period. Surface types are snow-covered and 
snow- free land, open water, and sea ice. Cloud layers are classified as low, 
middle, and high, where levels are defined by AVHRR channel 4 temperatures as 
follows: low cloud - > 265 K, middle cloud - 245-265 K, high cloud - < 245 K. 
These class maps can be manually produced using an image analysis system, where 
"objects" may take on any shape and size, or they can be generated 
automatically. Both methods were tested and, for the purposes of algorithm 
validation, produced similar images. Due to the relative ease of automatically 
generating the synthetic images, this method was followed. 

When the class maps are automatically generated, the minimum and maximum 
allowable sizes of surface objects for the first day, and cloud objects for each 
day are specified. An object is generated whose x,y dimensions are randomly 
chosen (uniform random number generator, URNG) within the restricted range, and 
the class of the object is randomly assigned (URNG). 
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Regions were then filled with data for each AVHRR and SMMR channel and for 
sea ice concentration. This filling was done with empirically- derived 
statistics. Data were based on class characteristic values (means and standard 
deviations) computed from training areas. Data values for each pixel in each 
channel were produced using a random number generator, the most important being 
one that produces normally distributed random numbers (NRNG) . Each 
artificially-generated element of class j is a vector, v^, of d features: 

V i “ Pi + ^3 

where ^ is the class mean vector of length d, 0 is a vector of random deviations 
for each feature selected from the multivariate Gaussian distribution of 
deviations, Aj is the lower triangular matrix decomposed from the d x d class 
covariance matrix, Ej (which is symmetric and positive definite), such that 

aa t - Sj 

The values from the Gaussian random number generator have a zero mean and unit 
variance and are constrained to be in the range of -3 to +3 which include 
approximately 99% of the data in a normally distributed population. Random 
number generators that produce values following uniform, negative exponential, 
and lognormal distributions are also possible. 

The surface map for the first day and cloud maps for all days were created 
with the above procedure. The surface maps for the third and fifth days, 
however, were modified versions of the first day. Snow and ice pixels were 
allowed to melt into land and water, respectively; ice pixels may advance into 
open water areas and snow may fall on land. The evolution was designed such 
that approximately 68% of the decisions resulted in an unchanged local area and 
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32% of the decisions resulted in either an advance or a retreat. Note that this 
adjustment produces non- rectangular surface regions. 

Synthetic data sets provide a model of the world observed through AVHRR 
and SMMR sensors. The model can be developed as simple geometrical shapes with 
no internal spatial and temporal variation or as free -form shapes with fuzzy 
edges (possibly fractal), texture, variable cloud properties and emissivities , 
spatially or temporally autocorrelated spectral properties, and so on. An 
intermediate level of complexity was chosen here, where the main concern was 
spatial and temporal variability of clouds and surfaces. 

2.5 Testing and Algorithm Comparison 

Four versions of the ISCCP algorithms are compared. The basic algorithm 
(abbreviated O-ISCCP) developed for low latitude summer conditions 1 recognizes 
only two surface types: land and water. No SMMR or sea ice concentration data 
are employed. Spatial/temporal tests in the initial classification step are 
thermal only (AVHRR channel 4). A bispectral threshold test (channels 1 and 4) 
is used as the final classif ication. This version with a thermal-only threshold 
test was also used to simulate winter applications (OT-ISCCP). The algorithm 
with modifications decribed above was the third version tested (M-ISCCP). The 
primary differences are in the recognition of multiple and changing surface 
types, and the inclusion of AVHRR channel 3 in the spatial/temporal tests, 
statistical tests, and in the threshold tests. Finally, the modified algorithm 
was used with a channel 3 and 4 difference threshold to identify clear pixels 


x This is not the identical algorithm applied by William Rossow at NASA, but 
rather a separate implementation which follows the same basic steps. 
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rather than the spatial/temporal tests (E-ISCCP). With this method, if the 
difference between the two channels is less than 4.5 K, the pixel is considered 
clear. If the difference is between 4.5 and 6.0 K, it is labeled "undecided”, 
otherwise it is labeled as cloud. This method is based in part upon the work 
of Oleson and Grassl (1985) . 

Four regions from the AVHRR imagery and four regions from the synthetic 
data sets are used as test data. Each region is 50 x 50 pixels or (250 km) 2 and 
differs in surface and cloud types and proportions. The synthetic data set 
image contains surface areas with 250 to 500 km as the minimum dimension 
("objects” are rectangular). Cloud sizes and distributions changed from day to 
day, with the minimum dimension ranging from 20 to 300 km. Surface proportions 
changed in both data sets by up to 20%. These changes were due to sea ice 
movement and melting. The surface/cloud types and proportions for each region 
are given in Table 3. 

Cloud fractions computed by each algorithm for each region and day are 
given in Tables 4 and 5. Also given in the table are the number of clear pixels 
used in the compositing step for each region and version. The actual cloud 
amount is shown for synthetic data sets, determined by counting the number of 
pixels in the region assigned to a cloud category. A manual interpretation was 
done using the image analysis system with all AVHRR, SMMR, and SMMR-derived data 
sets available. However, this value is only an approximation, and results from 
the algorithms may be more accurate. When cloud amounts are small to moderate, 
the manual interpretation generally overestimates cloud fraction. When cloud 
amounts are high, the manually interpreted cloud fraction is more accurate, but 
may slightly underestimate cloud fraction. 
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Table 3 

Percentages of surface and cloud types within the eight test regions. Cloud 
data are given for the middle five days of the seven day analysis period. 
Cloud categories are low, middle, and high. Surface types are land (L) , water 
(W) , ice (I), and snow/ice cap (S). 


Region: day 


Synthetic Data 
Low, Middle, High(%) 


AVHRR Data 
Low, Middle, High(%) 


1: 2 

4, 28, 39 

0 , 0 , 

3 

3 

24, 43, 6 

72, 18, 

0 

4 

47, 41, 11 

14, 0, 

7 

5 

6, 0, 81 

10, 20, 

16 

6 

58, 8, 33 

52, 10, 

33 

Surface : 

W(100) 

1(100) 


CM 

CM 

29, 0, 58 

12, 12, 

75 

3 

21, 42, 17 

13, 30, 

53 

4 

30, 12, 16 

27, 41, 

30 

5 

0, 17, 78 

18, 43, 

38 

6 

55, 7, 37 

18, 80, 

0 

Surface : 

L(35) , W(65) 

W(100) 


3: 2 

0, 33, 0 

10, 17, 

53 

3 

24, 20, 24 

17, 33, 

10 

4 

59, 20, 21 

10, 45, 

25 

5 

0, 0,100 

0, 20, 

3 

6 

20, 25, 16 

15, 40, 

20 

Surface : 

W(24), S ( 7 6 ) 

L(80) , W(1J 

4: 2 

0, 52, 0 

5, 10, 

82 

3 

31, 14, 54 

35, 30, 

30 

4 

12, 0, 28 

71, 10, 

0 

5 

10, 20, 51 

28, 25, 

7 

6 

48, 18, 17 

17, 50, 

23 


Surface : 


L(22), W(12) , 1(66) 


W(45), S (27) , 1(22), L(6) 
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Table 4 

"Actual" cloud fraction for synthetic data and cloud fraction computed by the 
four versions of the ISCCP algorithm. Values are for each of the middle five 
days of an analysis period. Cloud fraction computed by the bispectral threshold 
step is first given; cloud fraction determined after the initial classification 
is given in parentheses. The number of clear pixels in the compositing step is 
also shown. See text for version symbols. 


Algorithm 


Region: day 


1: 2 

3 

4 

5 

6 

# clear: 


2 : 


2 

3 

4 

5 

6 


# clear: 


3: 


2 

3 

4 

5 

6 


# clear: 


3 

4 

5 

6 


Actual 

O-ISCCP 

OT-ISCCP 

M-ISCCP 

E-ISCCP 

71 

76 (76) 

76 (76) 

78 (75) 

71 (75) 

73 

80 (84) 

80 (84) 

75 (71) 

73 (73) 

99 

100 (68) 

98 (68) 

100 (68) 

100 (100) 

87 

90 (93) 

90 (93) 

90 (90) 

88 (86) 

99 

100 (85) 

98 (85) 

100 (69) 

100 (100) 


643 

643 

125 

1581 


87 

93 (100) 

92 (100) 

94 (85) 

90 

(88) 

80 

97 (100) 

97 (100) 

82 (82) 

80 

(79) 

58 

83 (100) 

80 (100) 

77 (75) 

66 

(59) 

95 

100 (100) 

100 (100) 

95 (92) 

95 

(95) 

99 

100 (100) 

98 (100) 

100 (66) 

100 

(100) 



219 

219 

71 

1691 


33 

82 (56) 

78 (56) 

83 (46) 

79 (34) 

68 

94 (76) 

91 (76) 

94 (65) 

93 (68) 

100 

100 (67) 

94 (67) 

100 (70) 

100 (100) 

100 

100 (98) 

100 (98) 

100 (96) 

100 (100) 

61 

96 (97) 

90 (97) 

97 (87) 

96 (59) 



415 

415 

425 

2326 


52 

90 (67) 

76 (67) 

58 (62) 

76 (57) 

99 

99 (100) 

98 (100) 

99 (100) 

98 (100) 

40 

79 (83) 

66 (83) 

54 (53) 

65 (44) 

81 

98 (98) 

95 (98) 

81 (93) 

93 (88) 

83 

96 (96) 

88 (96) 

92 (82) 

92 (90) 

- _ 

420 

420 

32 

2105 


# clear: 
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Table 5 

Cloud fraction for actual data as computed by manual interpretation and four 
versions of the ISCCP algorithm. Abbreviations are the same as Table 5.2. 


Algorithm 


Region:day Manual 


1: 2 3 

3 90 

4 21 

5 46 

6 95 

# clear: 

2:2 99 

3 96 

4 98 

5 99 

6 98 


0-ISCCP 0T- ISCCP 


56 ( 5) 2(5) 
99 (50) 78 (50) 
70 (6) 12 ( 6) 
90 (44) 65 (44) 
99 (61) 89 (61) 

2404 2404 


99 (100) 99 (100) 

97 (100) 81 (100) 

94 (68) 75 (68) 

100 (100) 99 (100) 

92 (100) 73 (100) 


M-ISCCP 

E- ISCCP 

1 ( 6) 

2 (12) 

79 (39) 

93 (100) 

10 ( 6) 

17 (85) 

55 (42) 

56 (97) 

85 (57) 

98 (100) 

2264 

2327 


100 

(98) 

99 

(99) 

98 

(88) 

98 

(99) 

96 

(58) 

94 

(78) 

100 

(99) 

100 

(100) 

98 

(83) 

95 

(87) 


# clear: 


159 159 


1000 


3: 


2 

3 

4 

5 

6 


# clear: 


4: 


2 

3 

4 

5 

6 


80 

83 (78) 

75 (78) 

85 (72) 

83 (90) 

60 

62 (54) 

55 (54) 

61 (53) 

57 (64) 

80 

83 (87) 

73 (87) 

80 (80) 

80 (85) 

23 

31 (27) 

18 (27) 

29 (23) 

23 (27) 

75 

77 (79) 

72 (79) 

77 (76) 

75 (86) 

. _ 

1509 

1509 

972 

2129 


97 

100 (100) 

99 (99) 

100 (100) 

100 (100) 

97 

99 (65) 

63 (65) 

98 (58) 

97 (58) 

81 

95 (52) 

46 (52) 

85 (44) 

79 (82) 

60 

76 (48) 

51 (48) 

75 (51) 

65 (63) 

90 

95 (79) 

80 (79) 

92 (85) 

91 (91) 


1031 

1031 

362 

1594 


# clear: 
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Haze is difficult to detect with the image analysis system. While this 
condition is detected by the difference between channels 3 and 4, it is still 
difficult to observe in a color composite because the temperature and albedo are 
both very near those of the underlying surface. Visual detection requires an 
examination of actual brightness temperatures and albedos, and a comparison with 
the same area under other conditions. Of course, this is exactly what the ISCCP 
algorithm is designed to do. 

All versions of the algorithm performed best over land and water. Snow 
and ice remain the problem areas although the modified versions performed best 
under these conditions. When cloud amounts are high (more than 80%), all 
versions compute cloud fraction to within approximately 5%. When cloud amounts 
were low, the modified versions are more accurate, although cloud fraction is 
often too high. In the actual data, this is at least in part due to errors in 
the manual interpretation, as described above. In the synthetic data, this is 
probably due to the fact that clear sky areas are filled with values in the 
range of the mean plus or minus three standard deviations (following a Gaussian 
probability function) , so that extreme values may be beyond threshold cutoffs 
and will consequently be labeled as cloud. 

The basic versions often overestimate cloud amount by up to 20%. This is 
particularly true over ice where, in the bispectral threshold test, the 
threshold for water is used. This albedo threshold is too small to account for 
variation in sea ice albedos, and consequently many clear pixels are mistaken 
for cloud. This can be seen for region 1, day 2 of the actual data. Similar 
observations were made by Rossow (1987). A related situation is that the basic 
version often makes an accurate assessment of cloud fraction, but for the wrong 
reason. Day 3 of the same region and data set had over 70% of the cloud cover 
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as very thin cloud, possibly haze. Channels 1 and 4 alone cannot normally 
detect this condition, yet the threshold determined cloud amount for O-ISCCP is 
similar to the manually- determined amount. Here again, albedo contributions 
from the thin cloud are insignificant, so that the algorithm is calling "cloud" 
by the threshold step what it sees in channel 1 as sea ice. The snow and ice 
data sets used in the modified versions solve these problems by providing 
appropriate thresholds . 

An examination of the fraction of the region labeled "undecided" in the 
inital classification revealed that this value was smaller for the modified 
versions and that the difference between their initially classified cloud 
fraction and the threshold-determined cloud fraction was also smaller. This 
indicates that these versions are generally more stable. However, the number 
of clear pixels used in the compositing step was quite small for M-ISCCP, 
indicating that the three-channel spatial/temporal tests are perhaps too 
restrictive, as noted also in the previous chapter. Additionally, there were 
a few sequences where most pixels in a region were clear every other day, so 
that the temporal variation tests would never label these pixels as clear. 

The channel 3 minus 4 initial classification method proved to be 
problematic under certain cloud conditions. There were many occurrences of 
optically thick clouds with similar brightness temperatures in both channels 
which seemed not to be limited to a narrow range of temperatures. These 
conditions were initially classified as clear, and may or may not have been 
reclassified by the statistical tests. Therefore, the spatial/temporal 
classification tests - although not without problems - are more reliable for the 
actual data. Olesen and Grassl (1985) used AVHRR channel 5 in conjunction with 
the channel 3,4 difference, where the surface temperature was assumed to be 
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higher than that of the cloud. The two-dimensional histogram was then examined 
and partitioned into surface and cloud layers. This scheme would often fail 
with polar data since the surface temperatures are similar (and in winter 
usually lower) than low cloud temperatures due to ground-based inversions. 

Winter conditions in the Arctic can be particularly difficult to deal with 
algorithmically for two major reasons: strong surface inversions are the norm 
and no visible data are available. It is not unusual to find that the coldest 
object in the image is the surface. The data for study area 3 (January 6-12, 
1984) is no exception, where almost all cloud layers are warmer than the 
surface. Even over open water cold air advected from the adjacent ice creates 
weak lapse rates and/or inversions. Surface temperatures over land were 225- 
235 K, ocean (open and thin ice) in the southern portion of the image was 260- 
275 K, sea ice was 231-235 K, and clouds ranged from 215 to 258 K. 

To apply the ISCCP algorithm, additional modifications were necessary. 
The spatial variation test - where the warmest pixel in a subregion is 
considered clear - is certainly not valid. Therefore, this step was eliminated 
and the temporal variation test was used alone for the initial classification. 
The assumption that clouds are colder than the surface is found again in the 
final threshold step. This was modified so that a difference from the clear sky 
composite value in either direction signals a cloudy pixel. Finally, 
temperatures within the broadly-defined surface classes varied considerably 
across the image, in particular for snow/ice cap over Greenland (elevation 
change) and open water from the Norwegian Sea northward to Spitsbergen. Class 
characteristic values were no longer reliable, so statistical tests for cloud 
contamination were based only on the range of the extremum. 
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3 . 0 Summary 

Modifications to the basic ISCCP algorithm for cloud mapping over polar 
surfaces are proposed. The modified algorithm (M-ISCCP) is expected to yield 
an improvement in accuracy (estimated to be 5-10%) in computed cloud amount over 
the original, thermal-only version, depending on surface type and cloud 
proportions. This claim is for Arctic summers, from 60°N latitude to the pole, 
and is based on the tests results presented above, proportions of surface types 
over the Arctic, and average cloud amounts (see Section 1.2.4 and related 
references). All versions perform best over snow-free land and open water, so 
that improvement will be greater than this figure over snow, ice cap, and sea 
ice and less over open water and snow- free land. The most useful modification 
to the basic algorithm seems to be the ability to deal with additional surface 
types (i.e., sea ice and snow/ice cap) which allows a more appropriate choice 
of thresholds . 

Unfortunately, extracting, calibrating, and registering three or more 
AVHRR channels for seven days , two SMMR channels for each of three days , 
calculating sea ice concentration for three days , and developing a land/ice cap 
mask is not a trivial undertaking, so that this procedure is cost-effective only 
in areas where the more basic methods fail; i.e., over snow and ice. Previous 
studies (e.g. Rossow, 1987) have used only orbital AVHRR visible and thermal 
data. 

Persistent cloud cover and the small temperature difference between low 
clouds and water, ice, and snow remain the most difficult problems to solve 
given the basic structure of the ISCCP algorithm. To reduce the number of 
misclassifications resulting from these situations, channels 1 and 3 were 
incorporated. Unfortunately, channel 1 and the reflected component of channel 
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3 are not available for wintertime data. However, d'Entremont (1986) and Olesen 
and Grassl (1985) have shown that the addition of channel 3 improves the 
detection of low clouds and fogs at night, and also detects thin cirrus more 
confidently than any single-window infrared data, so that improvement could be 
expected with its use. The difference between channels 3 and 4 aids in the 
detection of optically thin cloud. 

The initial classif ication step is the most difficult part of the 
algorithm to refine due to its sensitivity to changes in thresholds. Error will 
propagate from this point, so it is important that all pixels labeled "clear" 
in this step actually are clear, but it is also important to obtain as many 
clear pixels as possible. The spatial/temporal tests are generally accurate, 
but fail in the case where a pixel is cloudy every other day. Using the simple 
channel 3 and 4 difference test for clear sky pixels was found to ease the 
computational burden and increase accuracy, but it fails for some middle -and 
low-level optically thick cloud decks. Under persistent cloud cover, the 5 -day 
compositing period may not be long enough to obtain an adequate sample of clear 
pixels. An examination of winter polar data undertaken here indicates that 
thermal -only spectral features are helpful, but alone are not sufficient unless 
examined over time. 

In summary, the best method of cloud detection includes first an accurate 
identification of surface types and changes. This allows thresholds to be set 
appropriately. Next the temporal variability of pixel radiances must be 
examined, using channel 4 and the reflected component of AVHRR channel 3 during 
summer, and the difference between channels 3 and 4 in conjunction with channels 
4 or 5 for winter analyses. Temporal changes are most important in winter, as 
surfaces may be colder than cloud layers and spectral information alone is 
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inadequate. Compositing over a 5 -day period, using 30 -day values where 
necessary, provides the clear sky information for the final bispectral 
thresholding of the daily data. Channels 1, 3 (reflected), and 4 may be used 
with summer data, and channels 3 (or 3-4) and 4 (or 5) should be used for winter 
analyses . 

Finally, the lack of 'ground truth' makes testing and validation 
difficult. Synthetic data sets provide a model of the world as observed through 
AVHRR and SMMR sensors. The model can be developed as simple geometrical shapes 
with no internal spatial and temporal variation or as free -form shapes with 
fuzzy edges (possibly fractal), texture, variable cloud properties and 
emissivities , autocorrelated spectral properties, and so on. The degree to 
which they model reality tests our understanding of the theoretical and 
empirical nature of the systems and how well we can couple these approaches into 
a precise description of the world. An intermediate level of complexity was 
chosen here; the main concern being spatial and temporal variability of clouds 
and surfaces. They provide a data set with known characteristics that allows 
performance to be quantified. 

4.0 Work Planned for the Last Six Months 

In the last six months of the second (and last) project year, work will 
continue on the ISCCP algorithm testing and modification for use in the polar 
regions. As described in this report, a number of problems with the application 
of this algorithm remain unsolved. In addition, a broader cloud analysis 
procedure will be developed from the modified ISCCP algorithm, where clouds are 
first detected and parameterized on the pixel scale, and cloud patterns and 
types are then be determined on a regional, (250 km) 2 scale. The ultimate goal 
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is to automate further and objectify the process of developing polar cloud 
climatologies for the purpose of monitoring climate change. Some background 
work on spring and summer Arctic cloud types has been done through a combination 
of manual and digital techniques by Kukla (1984), Robinson et al. (1986), Barry 
et al. (1987), and McGuffie et al. (1988). 





t % 


30 


References 

Barry, R.G. and R.G. Crane, 1985. Synoptic cloud climatology of the Arctic 
Ocean. Abstract Digest, Science Review of the NASA Climate Research 
Program, Science and Technology Corp., Hampton, VA: 51. 

Barry, R.G. A. Henderson- Sellers , and K.P. Shine, 1984. Climate sensitivity and 
the marginal cryosphere, in Climate Processes and Climate Sensitivity . J. 
Hansen and T. Takahashi (eds.), Geophys. Monog. 29, Amer. Geophys. Union, 
Washington, DC, pp. 221-337. 

Barry, R.G. , R.G. Crane, A. Schweiger, and J. Newell, 1987. Arctic cloudiness 
in spring from satellite imagery, J. Climatol., 7, 423-451. 

Crane, R.G. and R.G. Barry, 1984. The influence of clouds on climate with a 
focus on high latitude interactions, J. Climatol., 4, 71-93. 

Garand, L. , 1988. Automated recognition of oceanic cloud patterns. Part I: 
methodology and application to cloud climatology, J. Climate, 1, 20-39. 

Kukla, G.J., 1984. Variation of Arctic cloud cover during summer 1979, Part 1, 
Technical Report LDGO-84-2, Lamont- Doherty Geological Observatory of 
Columbia University. 

Olesen, F. and H. Grassl, 1985. Cloud detection and classification over oceans 
at night with NOAA-7, Int. J. Remote Sensing, 6 (8), 1435-1444. 

Raschke, E. , 1987. "Report of the ISCCP workshop on cloud algorithms in the 
polar regons," National Institute for Polar Research, WCP-131, WMO/TD-No. 
170, Tokyo, Japan, 19-21 August 1986. 

Rossow, W.B. , 1987. "Application of ISCCP cloud algorithm to satellite 

observations of the polar regions," in Raschke, E. , 1987. "Report of the 
ISCCP workshop on cloud algorithms in the polar regons," National 
Institute for Polar Research, WCP-131, WMO/TD-No. 170, Tokyo, Japan, 19-21 
August 1986. 

Rossow, W.B., 1987. "Preliminary documentation for ISCCP Cl data set", Pilot 
Climate Data System, Goddard Institue for Space Studies, NASA. 

Rossow, W.B., F. Mosher, E. Kinsella, A. Arking, M. Desbois, E. Harrison, P. 
Minnis, E. Ruprecht, G. Seze, C. Simmer, and E. mith, 1985. "ISCCP cloud 
algorithm intercomparison," J. Clim. Appl. Meteor., 24, 877-903. 

Schweiger, A., 1987. Retrieval of snow cover parameters using passive 

radiometry in the Federal Republic of Germany, Unpublished Master's 
thesis, Department of Geography, University of Colorado, Boulder. 

Serreze, M.S. and R.G. Barry, 1988. Synoptic activity in the Arctic Basin, 
1979-85, J. Climate (in press). 

Vowinckel, E. , 1962. Cloud amount and type over the Arctic, Publications in 



• "■* 


31 

Meteorology, No. 51, Montreal: Arctic Meteorology Research Group, McGill 
University, p. 27. 

Vowinckel, E. , and S. Orvig, 1970. The climate of the North Polar Basin, in 
Climates of the Polar Regions . S. Orvig, ed. , World Survey of Climatology, 
14, Elsevier, Amsterdam, 370 pp. 

WMO, 1987. Report of the ISCCP workshop on cloud algorithms in the polar 
regons, World Climate Research Programme , WCP-131, WMO/TD-No. 170, Tokyo, 
Japan, 19-21 August 1986. 



Publications Supported in Whole or Part 
by NAG- 5 -898 


32 


> 


Barry, R.G. , Crane, R.G. , Newell, J. f and Schweiger, A., 1988. "Arctic 

cloudiness in spring from satellite imagery. A response," J. Climatol., 
8, 539-540. 

Crane, R.G. and Barry, R.G. , 1988. "Comparison of the MSL synoptic pressure 
patterns of the Arctic as observed and simulated by the GISS General 
Circulation Model , " Meteor. Atmos. Physics, in press. 

Key, J., Maslanik, J.A., and Barry, R.G. "Cloud classification using a fuzzy 
sets algorithm: a polar example," Int. J. Remote Sensing, in press. 

Key, J.R., 1988. "Cloud analysis in the Arctic with combined AVHRR and SMMR 
data," unpublished Ph.D. dissertation, University of Colorado, Boulder, 

180 pp. 

Maslanik, J.A., Key, J.R., and Barry, R.G. "Merging AVHRR and SMMR data sea ice 
and cloud analysis," Int. J. Remote Sensing, in press. 

Maslanik, J.A., Barry, R.G. , and Crane, R.G. , 1987. "Approaches to the 

interpretation of cloud cover in the polar regions," (abstract). Clouds 
in Climate II . (A WCRP Workshop on Modeling and Observations, Columbia, 
MD). WCRP, NASA, NSF, NOAA, and DOE, p. 169. 

McGuffie, K. , Barry, R.G. , Newell, J., Schweiger, A., and Robinson, D. , 1988. 
"Intercomparison of satellite-derived cloud analyses for the Arctic Ocean 
in spring and summer," Int. J. Remote Sensing, 9 (3), 467-477. 







» * ■ -i 


ISCCP CLOUD DETECTION (BASIC) 


THERMAL DATA 
7 DAYS 
(250 km) 2 


VISIBLE DATA 
7 DAYS 


--> 


--> 



INITIAL CLASSIF. ; 
CLOUD/CLEAR/UNDECIDED ; 
THRESHOLDS FOR EACH 
SURFACE 


STATISTICAL TESTS 
OF 'CLEAR' PIXELS 


FINAL CLASSIF. OF 
EACH PIXEL AND 
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I 

Fig. 2. Flow chart of the basic ISCCP algorithm. Input are shown on the left; 
additional details are given on the right. 
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Fig. 3. Cloud cover and surface types for a single day and surface types for 
the same area five days later. Area covered is (1500 km) 2 . The combined 
AVHRR/SMMR data set provides for the mapping of cloud over varying surfaces, as 
shown here for changing ice extent northeast and south of Novaya Zemlya. 









